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ABSTRACT 


Gaining  a  detailed  and  thorough  understanding  of  the  modern  battle  space  is  vital  to  the  success  of  any 
military  operation.  Military  commanders  have  access  to  significant  quantities  of  information  which  originate  from 
disparate  and  occasionally  conflicting  sources  and  systems.  Combining  this  information  into  a  single,  coherent 
view  of  the  environment  can  be  extremely  difficult,  error  prone  and  time  consuming. 

In  this  paper  we  describe  the  Naval  Research  Laboratory’s  Virtual  Reality  Responsive  Workbench  (VRRWB) 
and  Dragon  software  system  which  together  address  the  problem  of  battle  space  visualization.  The  VRRWB  is 
a  stereoscopic  3D  interactive  graphics  system  which  allows  multiple  participants  to  interact  in  a  shared  virtual 
environment  and  physical  space.  A  graphical  representation  of  the  battle  space,  including  the  terrain  and  military 
assets  which  lie  on  it,  is  displayed  on  a  projection  table.  Using  a  six  degree  of  freedom  tracked  joystick,  the  user 
navigates  through  the  environment  and  interacts,  via  selection  and  querying,  with  the  represented  assets  and  the 
terrain. 

The  system  has  been  successfully  deployed  in  the  Hunter  Warrior  Advanced  Warfighting  Exercise  and  the 
Joint  Countermine  ACTD  Demonstration  One.  In  this  paper  we  describe  the  system  and  its  capabilities  in  detail, 
discuss  its  performance  in  these  two  operations,  and  describe  the  lessons  which  have  been  learned. 

KEYWORDS:  interactive  graphics,  workbench,  battle  space  visualization,  virtual  reality,  user  interface. 
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ABSTRACT 


Gaining  a  detailed  and  thorough  understanding  of  the  modern  battle  space  is  vital  to  the  success  of  any 
military  operation.  Military  commanders  have  access  to  significant  quantities  of  information  which  originate  from 
disparate  and  occasionally  conflicting  sources  and  systems.  Combining  this  information  into  a  single,  coherent 
view  of  the  environment  can  be  extremely  difficult,  error  prone  and  time  consuming. 

In  this  paper  we  describe  the  Naval  Research  Laboratory’s  Virtual  Reality  Responsive  Workbench  (VRRWB) 
and  Dragon  software  system  which  together  address  the  problem  of  battle  space  visualization.  The  VRRWB  is 
a  stereoscopic  3D  interactive  graphics  system  which  allows  multiple  participants  to  interact  in  a  shared  virtual 
environment  and  physical  space.  A  graphical  representation  of  the  battle  space,  including  the  terrain  and  military 
assets  which  lie  on  it,  is  displayed  on  a  projection  table.  Using  a  six  degree  of  freedom  tracked  joystick,  the  user 
navigates  through  the  environment  and  interacts,  via  selection  and  querying,  with  the  represented  assets  and  the 
terrain. 

The  system  has  been  successfully  deployed  in  the  Hunter  Warrior  Advanced  Warfighting  Exercise  and  the 
Joint  Countermine  ACTD  Demonstration  One.  In  this  paper  we  describe  the  system  and  its  capabilities  in  detail, 
discuss  its  performance  in  these  two  operations,  and  describe  the  lessons  which  have  been  learned. 

KEYWORDS:  interactive  graphics,  workbench,  battle  space  visualization,  virtual  reality,  user  interface. 


1  INTRODUCTION 


Gaining  a  detailed  tactical  picture  of  the  modern  battle  space  is  vital  to  the  success  of  any  military  operation. 
This  picture  is  used  to  direct  the  movement  of  assets  and  materiel  over  rugged  terrain,  day  and  night,  in  uncertain 
weather  conditions,  taking  account  of  possible  enemy  locations  and  activity.  To  provide  a  timely  and  accurate 
picture  of  the  battle  space,  most  modern  Command  Operations  Center  (COC)  have  access  to  a  multitude  of 

*  Corresponding  Author.  Effectiveness  of  Navy  Electronic  Warfare  Systems,  NRL.  Email:  durbin@nrl.navy.mil 
t Corresponding  Author.  ITT  Systems  &  Sciences.  Email:  julier@ait.nrl.navy.mil 

^B.  Colbert  is  with  ITT  Industries.  R.  King  and  J.  Crowe  are  with  Wagner  Associates.  B.  Doyle  is  with  the  Center  for  Computa¬ 
tional  Science,  NRL.  C.  Scannell  and  T.  King  are  with  the  Advanced  Information  Technology  Division,  Naval  Research  Laboratory. 
Zachary  Justin  Wartell  is  from  Georgia  Tech  and  Terry  Welsh  is  from  Iowa  State  University. 


systems  which  provide  information  from  many  different  sources  including  eye  witness  reports,  aerial  photographs, 
sonar  and  radar.  These  disparate  and  sometimes  conflicting  sources  must  be  combined  together  to  present  the 
tactical  view.  However,  the  quantities  of  information  are  sufficient  that  it  is  impossible  for  an  individual  to  be 
able  to  collect  and  comprehend  all  the  information.  Typically,  each  information  source  is  analyzed  individually 
by  a  specially  trained  technician.  The  data  sources  are  fused  to  give  an  overall  view  of  the  battle  space  which 
must  be  clear,  concise,  coherent,  complete  and  accurate.  However,  the  effectiveness  of  such  a  view  is  determined 
by  its  usability.  If  the  picture  contained  all  the  information  which  had  been  collected  the  commanders  would 
become  overloaded  by  the  quantitity  of  information.  Currently  most  tactical  decisions  are  made  using  detailed 
paper  maps  and  acetate  overlays.  Printing  and  distributing  these  materials  alone  can  take  several  hours. 

In  this  paper  we  argue  the  problem  of  information  overload  in  battle  space  awareness  can  be  largely  overcome 
using  interactive,  three  dimensional  computer  graphics.  By  filtering  the  information  which  is  displayed,  it  is 
possible  for  planners  to  examine  the  battle  space  across  many  levels  of  detail  from  a  broad  tactical  picture  to  the 
details  of  individual  units.  Intuitive  displays  and  interaction  paradigms  mean  that  it  is  possible  to  rapidly  and 
accurately  assess  the  situation.  We  describes  the  Naval  Research  Laboratory’s  solution  to  the  battlefield  visual¬ 
ization  problem  using  a  Virtual  Reality  Responsive  Workbench  (VRRWB)  and  the  Dragon  software  system.  The 
VRRWB  is  a  3D  graphics  system  which  allows  multiple  participants  to  interact  in  a  shared  virtual  environment 
and  physical  space.  A  graphical  representation  of  the  battle  space,  including  the  terrain  and  military  assets 
which  lie  on  it,  is  displayed  on  a  projection  table.  Through  a  careful  use  of  models,  user  interaction  metaphors 
and  information  displays,  the  system  presents  a  detailed  view  of  the  battle  space  without  overloading  the  user 
with  huge  quantities  of  information.  The  work  was  motivated  by  the  needs  of  the  Marine  Corps  Warfighting 
Laboratory  (MCWL).  In  July  1996,  NRL  was  asked  by  the  MCWL  to  field  a  prototype  situation  awareness  and 
planning  and  shaping  tool  for  use  in  the  Marine  Corp’s  Sea  Dragon  project4,12,13.  Since  its  participation  in 
SeaDragon’s  Hunter  Warrior  Advanced  Warfighting  Exercise  in  March  1997,  both  the  scope  and  functionality  of 
the  system  have  been  significantly  extended.  The  system  was  recently  deployed  at  the  Joint  Countermine  ACTD 
Demonstration  One  where  it  was  used  as  a  situation  awareness  tool.  Given  its  success  in  these  applications,  the 
workbench  has  proved  itself  to  be  a  valuable  tool  for  next  generation  COCs. 

The  structure  of  this  paper  is  as  follows.  The  problem  of  battle  space  awareness  is  discussed  in  Section  2 
and  it  is  argued  that  3D  graphics  with  a  high  degree  of  interactivity  is  a  very  effective  means  of  conveying  the 
necessary  information.  Section  3  describes  our  solution  which  consists  of  an  in-house  developed  software  system 
(known  as  “Dragon”)  that  is  controlled  by  a  user  with  a  responsive  workbench.  Since  the  requirements  of  Hunter 
Warrior  and  Joint  Countermine  have  largely  complementary  requirements,  we  briefly  describe  these  applications 
and  the  experiences  gained  by  the  Dragon  system  in  Section  4.  Future  work  is  described  in  Section  5.  The  paper 
is  summarized  in  Section  6. 


2  BATTLE  SPACE  AWARENESS 

2.1  The  Problem  of  Battle  Space  Awareness 

Battle  space  awareness  has  been  broadly  defined  by  Blanchard  as  “knowing  what  is  needed  to  win  with 
the  minimum  number  of  casualties”2.  This  is  an  extremely  broad  concept  and  Blanchard  identifies  seven  core 
concepts.  Of  these,  our  work  addresses  the  problems  of  terrain  awareness  (detailed  knowledge  about  the  form  of 
the  terrain  and  its  resources),  situation  awareness  (including  the  situations  of  friendly  and  enemy  forces  as  well 
as  weather)  and  mission  planning  and  rehearsal  systems  or  MPRS. 

The  twin  processes  of  acquiring  terrain  awareness  and  situation  awareness  can  be  very  difficult.  A  Combat 
Operations  Center  (COC)  receives  many  different  kinds  of  information  from  many  different  sources.  All  of  the 
different  reports  must  be  interpreted,  reconciled  and  fused  to  give  a  consistent  and  comprehensive  view  of  the 
battlespace.  We  believe  that  most  battle  space  awareness  systems  must  possess  the  following  capabilities: 


(a)  Output  from  a  JMCIS  terminal. 


(b)  Output  from  a  Workbench. 


Figure  1:  Two  and  three  dimensional  displays  of  a  simulated  scenario 


•  Present  a  comprehensive  and  timely  view  of  the  environment. 

•  Provide  a  dynamic  range  of  resolution  sufficient  to  track  units  ranging  from  aircraft  carriers  to  six-Marine 
fire  teams. 

•  Support  information  filtering.  In  other  words,  it  is  possible  to  display  a  selected  subset  of  available  infor¬ 
mation  using  a  particular  display  technique. 

•  Prioritize  events  and  issue  alarms. 

•  Represent  limitations  in  the  awareness.  For  example,  data  fusion  procedures  might  be  unable  to  classify  a 
potential  target  or  the  position  uncertainty  could  be  very  large. 

•  Provide  the  user  with  all  of  the  information  which  is  currently  available. 

•  Support  interaction  with  planning  and  simulation  systems  so  that  the  current  situation  and  future  plans 
can  be  presented  within  a  common  framework. 


2.2  The  Limitations  of  Two  Dimensional  Displays 

Historically,  war  fighters  have  used  maps  with  acetate  overlays  marked  with  grease  pencils  to  plan  and  update 
awareness  of  the  battle  space.  Such  methods  are  poorly  suited  for  the  modern  day  COC  which  receives  many 
megabits  of  data  per  second  from  many  different  types  of  data  sources.  The  concept  of  a  computerized  battle  space 
visualization  system  is  not  new.  The  Joint  Maritime  Command  Information  System  (JMCIS),  is  a  widely  fielded 
military  information  system  which  includes  a  visualization  module.  A  JMCIS  terminal  is  capable  of  displaying 
any  type  of  information  which  can  be  stored  within  the  JMCIS  system  including  the  position  and  classification  of 
entitites.  The  information  can  be  displayed  in  either  tabular  or  iconic  form.  Figure  1(a)  shows  a  typical  output. 
The  locations  of  approximately  six  entities  are  shown  as  icons  which  are  superimposed  on  top  of  an  ARC  Digital 
Raster  Graphic  (ADRG)  map.  Each  icon  is  color  coded  and  labeled  with  the  track  number.  The  map  provides 
detailed  information  about  the  terrain  including  its  topology  and  the  location  of  roads.  Users  can  navigate  by 
panning  and  zooming  through  the  environment.  Entities  can  be  selected  and  queried  using  a  mouse  and  pop 
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Figure  2:  Some  activity  during  the  Hunter  Warrior  Exercise. 


up  menus.  Although  JMICS  is  extremely  powerful  and  flexible,  it  has  two  significant  problems  for  information 
overload  and  battlefield  visualization  —  clutter  and  the  loss  of  three  dimensional  information. 

Clutter  arises  when  the  terminal  is  displaying  significant  quantities  of  information.  In  Figure  1(a)  several 
entities  lie  sufficiently  close  to  one  another  that  their  icons  and  track  names  overlap.  Furthermore,  the  details  on 
the  map  can  make  it  difficult  to  determine  the  positions  of  the  entities.  Some  of  these  difficulties  can  be  overcome 
by  changing  the  visibility  options  —  for  example,  replacing  the  map  by  a  monochrome  background  will  make  the 
positions  of  the  entities  clearer.  However,  these  solutions  reduce  the  amount  of  information  which  is  displayed. 

The  second  problem  is  that  JMCIS  only  provides  a  plan  view  of  the  battle  space.  However,  the  battle  space  is 
inherently  three  dimensional  and  understanding  the  relationship  between  the  terrain,  ground  assets,  aircraft  and 
flight  corridors  can  be  critical.  For  example,  in  the  scene  which  is  displayed  the  altitude  of  the  ground  varies  by 
over  a  thousand  feet.  In  a  two  dimensional  display  this  information  can  only  be  revealed  by  studying  the  contour 
lines  and  reviewing  altitude  information  for  each  entity. 

We  believe  that  these  difficulties  can  be  reduced  by  replacing  the  two-dimensional  plan-based  view  by  a  three 


dimensional  display. 


2.3  Three  Dimensional  Displays 

Three  dimensional  graphical  displays  replace  a  two  dimensional  plan  view  by  a  full,  three-dimensional  repre¬ 
sentation  of  the  scene.  Through  introducing  depth  into  the  environment,  such  displays  can  present  much  greater 
quantities  of  information.  This  can  be  appreciated  by  comparing  the  output  from  the  Dragon  system,  shown  in 
Figure  1(b),  with  that  from  the  JMCIS  terminal.  At  the  instant  shown  both  systems  are  displaying  parts  of  the 
same  environment  (the  29  Palms  Area  which  is  discussed  in  more  detail  in  Section  4) .  The  3D  display  represents 
the  terrain  as  a  contoured  surface  which  is  textured  with  an  ADRG  map.  The  form  of  the  terrain  can  be  directly 
observed  without  reference  to  the  contour  lines  on  the  map.  Entities  are  represented  by  icons  and  models  which 
lie  on  and  above  the  terrain.  Contrast  and  depth  cues  highlight  the  positions  of  these  entities  against  a  cluttered 
surface.  Finally,  the  user  has  much  greater  control  over  the  viewport  location  and  viewing  angle. 

Given  its  advantages,  three  dimensional  graphical  displays  have  been  used  for  a  number  of  years  in  MPRS 
applications.  Most  of  these  applications  are  “stealth  viewers”  —  the  user  is  a  passive  observer  who  can  navigate 
through  an  environment  and  study  a  simulation  from  many  different  vantage  points.  However,  these  systems 
cannot  meet  the  needs  of  battle  space  awareness  systems  for  two  main  reasons.  First,  the  types  of  data  created 
by  a  simulation  are  different  than  those  from  received  by  a  COC.  The  simulations  only  produce  a  few  kinds  of 
exact  data  with  no  ambiguities  or  uncertainties.  Second,  the  role  of  the  user  in  a  battle  space  awareness  system  is 
much  greater  than  that  in  a  stealth  viewer.  The  user  is  not  just  a  passive  observer  —  the  user  must  also  be  able 
to  play  an  extremely  active  role  in  querying  entities  and  changing  the  state  of  the  environment.  Virtual  reality 
(VR)  is  ideally  suited  for  this  type  of  task. 

Although  VR  is  usually  discussed  in  the  context  of  a  fully  immersive  system  with  a  head  mounted  display, 
the  term  is  fairly  broad  and  includes  a  number  of  different  paradigms11  which  can  be  classified  as  immersive, 
partially  immersive  or  non-immersive.  Immersive  VR  systems  use  head  mounted  displays,  push  booms  and 
immersive  rooms9  to  the  user  the  impression  of  being  immersed  within  a  synthetic  environment.  Non-immersive 
VR  systems  do  not  attempt  to  simulate  a  complete  environment.  The  battle  space  awareness  task  does  not 
require  that  command  and  control  staff  are  immersed  in  a  virtual  COC.  Rather,  the  types  of  tasks  which  are  to 
be  supported  are  those  which  are  conducted  at  a  desk.  An  ideal  system  for  working  with  such  tasks  is  the  virtual 
reality  responsive  workbench  (VRRWB)5, 6’ 8’ 10  which  is  described  in  the  next  section. 


3  THE  NRL  WORKBENCH  AND  DRAGON  SOFTWARE 

3.1  The  Virtual  Reality  Responsive  Workbench 
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The  main  components  of  a  VRRWB  are  illustrated 
in  Figure  3.  A  desk-side  workstation  renders  a  three 
dimensional  image  of  the  screen.  This  image  is  back- 
projected  onto  a  horizontally  mounted  screen.  Users 
stand  in  the  viewing  area  and  interact  with  the  bench 
through  a  number  of  input  devices  including  a  tracked 
three  dimensional  mouse,  a  pair  of  pinch  gloves  and  a 
speech  recognition  system. 

The  display  system  can  operate  in  a  monographic 
or  stereographic  mode.  In  the  stereographic  mode  the 
display  alternates  between  rendering  images  for  the  left 

and  right  eyes  on  each  frame  and  each  user  wears  a  pair  Figure  3.  The  Virtual  Responsive  Workbench 

of  LCD  shutter  glasses  giving  each  user  the  illusion  that  the  project  image  rises  above  or  sinks  below  the  surface  of 
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the  table.  The  perception  of  depth  enhances  the  perception  of  the  three  dimensional  structure  of  the  environment. 

The  workbench  is  suited  for  the  needs  of  battle  space  visualization.  It  creates  a  virtual  environment  ideally 
suited  for  fine  grained  manipulation  and  allows  a  number  of  users  to  collaborate  in  the  same  physical  and  virtual 
environments  at  the  same  time. 


3.2  Display  and  Symbology 


(a)  Symbology  used 


(b)  An  Annotated  Entity 


Figure  4:  Display  symbology  and  marker  types 

The  Dragon  system  presents  the  user  with  a  three  dimensional  image  of  an  abstraction  of  the  battle  space. 
Figure  2  shows  a  screen  capture  of  some  of  the  activity  encountered  in  the  Hunter  Warrior  AWE.  It  shows  many 
of  the  essential  features  of  the  Dragon  system  —  the  terrain  and  coast  line,  a  number  of  on-shore  and  off-shore 
military  assets,  the  user’s  laser  wand  (described  later)  and  information  windows  providing  data  about  units  and 
the  current  time.  Considerable  time  and  effort  was  devoted  to  developing  a  symbology  which  is  intuitive,  clear 
and  meets  the  situation  awareness  needs. 

The  terrain  contour  information  is  derived  from  a  Digital  Terrain  Elevation  Data  (DTED)  Level  1  database. 
This  data  is  used  to  construct  a  polygonal  skin  over  which  is  draped  a  texture  derived  from  an  ADRG  dataset.  The 
latter  is  simply  a  digital  form  of  the  standardized  maps  which  are  used  in  a  COC.  The  maps  include  the  military 
grid  system,  contour  lines,  range  markers,  and  other  import  designations  that  are  used  by  the  commanders  in 
their  day-to-day  tasks.  To  give  a  greater  appreciation  of  the  form  of  the  terrain,  it  is  possible  to  scale  the  map  in 
the  vertical  direction. 

The  Dragon  system  supports  over  200  different  models  of  military  assets,  some  of  which  are  shown  in  Fig¬ 
ure  4(a).  Since  a  standard  has  not  yet  been  established  for  three  dimensional  tactical  displays,  the  symbology  was 
designed  through  discussion  with  a  number  of  military  commanders.  Roughly  half  of  our  model  suite  is  obtained 
from  commercial  sources  and  consists  of  models  of  complex  equipment  such  as  tanks,  ships  and  helicopters.  The 
remaining  symbols  were  developed  inhouse.  Large  units  are  represented  by  flags  bearing  the  name  and  seal  as¬ 
sociated  with  that  unit.  Smaller  units  (platoons,  squads,  and  fire  teams,  for  example)  are  represented  by  simple 
cubes  textured  on  all  sides  with  standard  military  symbols  such  as  an  ‘X’  for  infantry  and  a  sideways  ‘E’  for 
engineers.  These  are  easily  recognizable  by  the  users.  Since  some  percentage  of  the  user  base  is  color  blind,  it 
was  necessary  to  use  a  combination  of  colors  and  symbols  to  differentiate  between  the  different  forces. 


An  important  question  is  to  decide  the  scale  of  the  models  which  are  displayed  on  the  terrain.  If  the  models 
are  too  small  they  can  only  be  seen  close  up  and  it  is  not  possible  to  obtain  an  overall  view  of  the  environment. 
Conversely,  if  the  models  are  too  large  then  the  results  can  be  misleading.  Entity  models  could  overlap  one 
another  (similar  to  the  difficulties  encountered  by  the  JMCIS  system).  Initially  the  entities  were  all  presented  at 
the  same  scale.  Our  current  system  scales  the  entities  between  one  and  fifty  times  real  size  on  demand.  We  plan 
to  investigate  a  system  which  smoothly  changes  the  entity  scale  as  the  user  zooms  in  towards  the  terrain7  and  an 
aggregation  system  which  will  group  entities  together  into  a  more  encompassing  symbol. 

It  is  possible  to  query  entities  or  the  terrain  for  further  information.  We  have  developed  two  approaches.  The 
first,  illustrated  in  Figure  2  is  a  pop  up  display  which  provides  information  about  the  entity  name,  type,  current 
status  and  other  pertinent  information.  The  second  means  of  displaying  information  is  to  attach  a  status  icon 
directly  to  the  entity.  This  capability  is  illustrated  in  Figure  4(b)  where  the  name  of  the  entity  is  displayed  as  a 
flag. 


3.3  Interaction  Methods 

Much  of  the  success  of  a  visualization  system  de¬ 
pends  on  the  ease  with  which  a  user  can  navigate 
through  the  virtual  environment  and  interact  with  ob¬ 
jects.  If  the  interaction  is  extremely  difficult,  perform¬ 
ing  tasks  can  be  stressful  and  inefficient.  These  is¬ 
sues  were  crucial  in  the  design  of  the  system,  where  it 
was  expected  that  users  would  be  operating  the  sys¬ 
tem  continuously  for  many  hours  at  a  time.  We  have 
experimented  with  three  methods  of  interaction  with 
the  Workbench:  gesture  recognition  with  a  pinchglove, 
a  spaceball,  a  mounted  joystick  mouse  and  a  three  di¬ 
mensional  mouse  which  projects  a  virtual  wand.  The 
wand  was  chosen  because  of  its  ease  of  use  (it  is  pos¬ 
sible  to  stand  in  one  position  and  select  and  highlight 
entities  on  any  part  of  the  visible  display)  and  physical 
robustness  (it  contains  few  moving  parts  and  does  not 
rely  on  precise  callibration) . 

Figure  5  shows  a  number  of  users  working  with  the 
bench  and  its  laser  wand.  The  metaphor  which  we  used  was  one  of  a  laser  pointer.  The  position  and  orientation 
of  the  pointer  is  tracked  using  an  electro-magnetic  tracker  and  a  virtual  ray,  pointing  in  the  direction  of  the  wand 
is  projected  out  onto  the  terrain.  The  wand  can  be  seen  in  Figure  2  as  a  semi  transparent  tube  which  runs  up 
from  the  bottom  of  the  screen. 

There  are  two  basic  tasks:  change  the  viewpoint  of  the  virtual  environment  and  perform  operations  on  the 
entities  within  the  environment.  We  have  introduced  two  complementary  metaphors  for  navigation  —  exocentric 
and  egocentric.  In  the  exocentric  metaphor,  the  user  remains  stationary  and  the  map  moves.  In  the  egocentric 
view,  which  is  akin  to  more  conventional  virtual  reality  navigation  systems,  the  map  remains  stationary  and  the 
user  “flies”  through  the  environment. 

The  exocentric  metaphor  follows  directly  from  the  way  in  which  a  user  interacts  with  a  real  physical  map 
placed  on  a  table  top  surface.  If  the  map  is  too  large  to  completely  fit  on  the  table,  the  user  must  move  the  map 
left,  right  and  up,  down  in  order  to  move  the  desired  portion  of  the  map  onto  the  table  top  surface.  In  the  Dragon 
system,  one  combination  of  buttons  on  the  joystick  provided  a  similar  capability.  A  user  presses  a  combination 
of  buttons  and  moves  the  joystick  parallel  to  the  surface  of  the  bench.  The  terrain  mimics  the  user’s  motion 
and  scrolls  left/right  or  up/down.  If  the  user  moves  the  joystick  away  from  the  surface  of  the  bench  the  scale  of 


Figure  5:  The  Virtual  Responsive  Workbench 


the  terrain  increased,  whereas  moving  the  joystick  towards  the  surface  of  the  table  causes  the  scale  to  decrease. 
Functionality  was  also  available  for  the  user  to  change  his  heading  and  pitch  in  the  environment.  Thus,  the  user 
was  able  to  manipulate  the  viewpoint  into  the  virtual  environment  very  easily  and  was  quickly  able  to  achieve 
any  desired  viewpoint. 

Although  the  exocentric  metaphor  is  suited  for  global  planning  tasks,  it  is  not  well  suited  for  those  operations 
in  which  some  degree  of  immersion  is  required.  For  example,  it  is  difficult  to  fly  over  the  terrain  or  position 
the  viewpoint  at  the  location  of  an  asset  on  the  terrain.  To  meet  these  requirements,  an  egocentric  navigation 
metaphor  was  implemented.  Under  this  metaphor,  the  user  moves  through  the  virtual  environment.  Since 
navigation  with  a  full  six  degrees  of  freedom  can  be  difficult,  we  have  implemented  three  navigation  modes.  The 
pan/zoom  mode  moves  the  user  in  the  direction  of  the  mouse  gesture.  The  pitch/yaw  mode  either  changes  the 
pitch  or  the  yaw  (heading)  one  is  viewing.  The  final  navigation  mode  is  rotate/zoom  to  make  it  possible  for  the 
user  to  user  to  rotate  around  a  specific  object  or  point  on  the  terrain.  It  is  also  possible  to  “tether”  the  origin  of 
the  viewpoint  to  an  entity.  If  the  entity  moves,  the  view  point  moves  by  the  same  amount  in  the  same  direction. 
User  studies  are  in  progress  to  determine  the  most  efficient,  naturual  and  easy  way  to  use  navigation  modes. 

The  user  interacts  with  all  entities  on  the  terrain  with  the  virtual  laser  wand.  An  entity  is  selected  by  pointing 
the  wand  at  the  entity  in  question.  As  shown  in  Figure  2,  the  highlighting  is  signified  by  drawing  a  wire  frame 
sphere  about  the  appropriate  entity.  Once  an  entity  has  been  selected,  the  user  can  perform  several  actions.  First, 
the  entity  can  be  queried  to  find  out  further  information  such  as  its  name,  track  number  or  status  information. 
This  basic  functionality  is  required  when  the  user  is  a  passive  observer.  When  the  user  plays  a  proactive  role  (in, 
for  example,  a  planning  application)  the  user  will  be  able  to  create,  pick  up,  move  and  place  entities  on  different 
parts  of  the  terrain.  By  pressing  a  button  on  the  joystick,  the  user  picks  up  the  selected  entity  and  moves  it 
around  the  virtual  environment.  Entities  suspended  above  the  terrain  project  a  red  drop  line  to  the  surface  to 
indicate  position  and  altitude.  Terrain-based  entities  such  as  tanks  or  ships  drop  to  the  terrain  surface  when 
released;  other  entities  (such  as  aircraft)  remain  at  the  same  position. 


3.4  Software  Architecture 


The  development  of  the  Dragon  system  intro¬ 
duced  a  number  of  challenges.  First,  it  receives 
information  from  many  different  data  sources  in¬ 
cluding  simulations  and  situation  awareness  sys¬ 
tems.  Each  data  source  has  its  own  character¬ 
istics  (the  types  of  data  it  provides,  the  queries 
which  can  be  made  and  the  update  rates)  and 
they  must  be  converted  into  a  common  format 
which  can  be  efficiently  stored  and  processed. 
Second,  it  must  be  possible  for  the  system  to 
render  a  large  and  complicated  graphical  scene 
with  an  interactive  frame  rate  (more  than  10 
frames  per  second).  To  meet  these  objectives,  we 
designed  the  system  whose  architecture  is  illus¬ 
trated  in  Figure  6.  Finally,  the  computing  power 
required  cannot  exceed  the  capacity  of  current 
desk  side  workstations.  The  system  consists  of 
two  key  components:  an  entity  manager  and  a 
rendering  engine  which  communicate  through  a 
high-speed  shared  memory  arena. 


Figure  6:  The  Architecture  of  the  Dragon  System. 


The  asynchronous  entity  manager  is  the  interface  between  the  external  data  systems  and  the  rendering  engine. 
It  collects  the  data  from  the  different  sources,  parses  it  to  a  common  representation  and  builds  a  table  of  current 


entities.  To  minimize  the  amount  of  information  which  flows  between  the  manager  and  the  Tenderer,  only  entity 
state  changes  are  communicated.  To  control  user  access  operations,  each  entity  has  a  set  of  permissions  associated 
with  it.  These  permissions  are  used  to  manage  the  different  needs  and  requirements  from  different  systems.  For 
example,  in  a  battlefield  awareness  situation  it  is  not  desirable  for  a  user  to  be  able  to  move  the  entities  which 
arise  from  external  data  feeds.  However,  in  a  planning  and  shaping  application  it  is  necessary  that  the  user  be 
able  to  move  them.  Currently,  the  entity  manager  accepts  input  from  GCCS-M,  DIS  and  persistent  data  storage. 
Proposed  data  systems  include  Entity  Manager  to  Entity  Manager  communications  (allowing  collaborative,  multi¬ 
user  distributed  systems)  and  distributed  intelligent  agent  architectures3  which  make  it  possible  to  interact  with 
agent  based  systems  such  as  QuickSet  or  multimodal  interaction  systems. 

The  rendering  engine  manages  all  user  interactions  and  has  two  key  roles.  The  first  is  an  input  function  — 
it  collects  the  data  from  the  tracking  devices  and  detects  whether  the  wand  is  intersecting  with  the  entities  or 
the  terrain.  Second,  it  renders  the  images  which  are  seen  by  the  user.  To  achieve  the  required  frame  rates  we 
developed  a  custom  Tenderer  which  is  built  on  top  of  Silicon  Graphic’s  Performer  library. 


4  SAMPLE  APPLICATIONS 

In  this  section  we  describe  two  applications  of  the  workbench  and  the  Dragon  software  —  the  Hunter  Warrior 
AWE  and  the  Joint  Countermine  Operational  Simulation  (JCOS).  Together,  these  applications  demonstrate  the 
system’s  capability  for  terrain  awareness,  situation  awareness  and  MPRS. 


4.1  Hunter  Warrior 

4.1.1  Operational  Objectives 

NRL  was  approached  in  July  1996  by  the  Marine  Corps 
Warfighting  Laboratory  to  produce  a  prototype  system  for  the 
Hunter  Warrior  Advanced  Warfighting  Experiment  (AWE)  which 
was  held  in  March  1997.  The  Hunter  Warrior  AWE  is  the  first 
of  a  series  of  three  AWEs  that  make  up  the  SeaDragon  process, 
a  program  to  explore  technology  and  its  roll  in  supporting  the 
Marines  in  the  21st  century.  NRL  delivered  the  first  version  of 
the  Dragon  software  to  MCTSSA,  Camp  Pendleton,  CA  during 
the  first  week  in  December  1996.  A  final  version  was  delivered 
in  mid-February  1997  and  included  a  near-realtime  interface  to 
import  a  Joint  Maritime  Command  Information  System  (JMCIS) 
data  feed. 


Figure  7:  Plan  View  of  29  Palms 


The  operational  portion  of  Hunter  Warrior,  shown  in  Figure  7  took  place  at  the  Twenty-nine  Palms  National 
Training  Center,  California.  For  this  exercise,  the  Marines  simulated  a  regional  conflict  between  two  neighboring 
countries,  one  of  which  had  coastal  port  facilities.  These  port  facilities  were  important  for  rapidly  off  loading 
large  equipment  to  support  the  Marines  landing.  Since  these  port  facilities  do  not  exist  in  the  real  environment 
(Twenty-Nine  Palms  is  in  the  high  desert  of  Southern  California),  the  terrain  dataset  was  modified  to  introduce 
an  artificial  coast  line. 


4.1.2  Results 

Despite  the  fact  that  the  delivered  system  was  a  prototype,  we  received  very  positive  comments  from  com¬ 
manders  and  technicians  alike.  Throughout  the  exercise  the  system  was  used  almost  continuously  and,  by  the 
end  of  the  exercise,  the  marines  were  using  the  workbench  in  preference  to  paper  maps  and  acetate  overlays.  In 


(a)  JCOS  in  operation 


(b)  Close  up  of  activity 


Figure  8:  Screen  shots  from  the  JCOS  exercise. 


addition  to  its  primary  role  as  a  situation  awareness  tool,  the  workbench  was  also  used  as  a  briefing  tool. 

Colonel  Wood,  Director  of  the  Marine  Corps  Warfighting  Laboratory,  wrote  “I  consider  the  capability  provided 
by  the  workbench  to  be  a  seminal,  critical  break-through  over  previous  technology  and  competing  systems.  This 
fact  became  even  more  obvious  as  we  demonstrated  its  capability  for  the  several  hundred  VIPs  who  all  commented 
on  the  workbench’s  tremendous  potential.” 


4.2  Joint  Counter  Mine  Operational  Simulation 

4.2.1  Exercise  Objective 

Between  the  18th  of  August  and  the  5th  of  September  1997,  Dragon  was  used  to  support  the  Joint  Countermine 
Operational  Simulation  (JCOS)  component  of  the  Joint  Counter  Mine  (JCM)  ACTD.  The  purpose  of  JCM  is  to 
measure  the  effectiveness  of  nine  novel  techniques  for  mine  clearance  in  combat  situations.  A  set  of  simulated 
exercises  were  conducted  at  Camp  Lejeune,  North  Carolina,  and  the  Dragon  system  was  used  as  a  viewer  to  aid 
in  the  analysis  of  the  archived  data.  Since  it  was  being  used  as  a  stealth  viewer,  users  were  not  able  to  change  the 
state  of  any  entities.  Although  this  was  less  demanding  from  the  perspective  of  entity  manipulation  techniques,  it 
introduced  a  number  of  new  challenges  with  the  navigation  modes,  labeling  and  annotations,  continuous  updates 
of  over  three  hundred  entities  and  the  ability  to  provide  real-time  animation  of  events  such  as  explosions  and 
smoke. 


4.2.2  Results 

Figure  8  shows  two  screen  shots  from  the  exercise.  Unlike  the  Twenty-Nine  Palms  Area,  Camp  Lejeune  is 
extremely  flat  (the  highest  point  in  the  data  set  is  a  man  made  hill  which  is  under  20m  in  height).  Therefore,  the 
problem  of  terrain  awareness  is  limited.  The  Workbench  was  situated  in  a  small  room  off  of  the  main  control  and 
briefing  area.  Almost  all  of  the  VIPs,  both  civilian  and  military,  commented  on  the  potential  for  the  workbench 
to  provide  a  thoroughly  integrated  visualization  environment  for  multiple  data  streams  that  would  greatly  assist 
the  user  in  performing  his  or  her  tasks. 


5  FUTURE  WORK 


The  experiences  which  we  have  obtained  from  this  work  suggests  that  there  are  many  further  areas  of  research 
and  development  to  explore.  Some  of  the  most  important  ones  are: 


•  Extend  the  size  of  the  terrain  dataset  which  can  be  represented  through  the  use  of  wide  area  database 
management  techniques. 

•  Increase  the  resolution  of  the  terrain. 

•  Increase  the  types  of  interaction  technology  which  can  be  used  with  the  workbench.  Such  technologies 
include  speech,  natural  language  recognition,  haptic  (force  feedback)  devices  and  tracked  laser  pointers. 

•  Investigate  the  visualization  of  different  types  of  data  using  different  display  technologies  including  tech¬ 
niques  such  as  overlays,  gauges,  color,  texture  and  sound. 

•  Develop  multi-user  workbenches.  These  are  capable  of  displaying  different  views  to  two  or  more  workers1. 
There  are  a  number  of  different  potential  uses  including  two  or  more  tracked  views  and  information  filtering. 

•  Provide  simple  representations  of  the  weather  to  take  into  account  the  effects  of  fog  and  cloud  cover. 


6  CONCLUSIONS 


Modern  day  COCs  are  flooded  by  gigabits  of  information  from  many  different  information  systems.  Military 
commanders  and  planners  must  filter  this  information  to  determine  a  consistent  and  accurate  picture  of  the  battle 
space  for  situation  awareness  and  MPRS.  However,  the  sheer  volume  of  this  raw  data  can  lead  to  information 
overload.  This  situation  will  only  become  worse  as  more  sources  of  data  become  available. 

In  this  paper  we  have  argued  that  the  problem  of  information  overload  can  be  overcome  through  a  combination 
of  carefully  designed  user  interfaces  and  visualization  displays.  These  make  it  possible  for  a  user  to  filter  the 
data  such  that  only  information  relevant  to  a  particular  stage  of  a  planning  process  is  displayed  as  required. 
The  workbench,  which  supports  the  paradigm  of  desk-based  work  with  multiple  collaborators,  is  ideally  suited 
to  the  problems  of  situation  awareness  and  MPRS.  The  performance  of  our  Dragon  software  system  has  been 
demonstrated  in  a  number  of  military  exercises. 
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